Systems and Methods for Digital Auditory Mapping

ABSTRACT

Methods and system for providing digital audio mapping are provided wherein data representing real or virtual worlds is obtained from internal or external data sources and used to generate a digital auditory map that can be navigated using auditory sound cues. The digital audio map can be created by a digital audio map generator and presented to the user through a digital audio map presentation interface. The digital audio map presentation interface can provide multiple navigation modes to the user, including a tree-based mode, a grid-based mode, and a first-person-based mode. The user is able to dynamically select the preferred navigation mode.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. ProvisionalApplication No. 63/052,645, filed on 16 Jul. 2020, incorporated byreference herein and for which benefit of the priority date is herebyclaimed.

FEDERALLY ENDORSED RESEARCH

Not applicable.

SEQUENCE LISTING OR PROGRAM

Code listing of a Javascript file is attached as Appendix 1.

FIELD OF INVENTION

The present invention relates to digital auditory maps. Specifically,the present invention relates to systems and methods for mapping real orvirtual environments for navigation using auditory cues.

BACKGROUND OF THE INVENTION

Access to nonvisual maps has long required special equipment andtraining to people with visual impairments. Currently, the main methodof nonvisual map representation is utilizing raised line tactilegraphics, which are labeled using braille, or an interactive system,such as a touch screen being placed under a paper map. However, atactile map is inconvenient and difficult to obtain, requiring a tactileembosser or other machine to create the graphic. There are currentlydevices that could potentially allow users to view a refreshable tactilegraphic, but these devices are expensive and not commercially available.One method of nonvisual representation that is ubiquitous and availableon almost all devices is audio. There has been some research aboutshowing map information completely in audio, however, existingapproaches only show one kind of data, such as Open Street Map data, donot have a way that allows users to easily recognize shapes, may notallow users to navigate the map on their preferred device, such as aniPhone, and only may show one of the below interface modes. Currentauditory maps may be provided in one of the following modes: 1)first-person mode where sounds are positioned at objects around thelistener and played using 3-Dimensional audio (in a loop), such that asa user changes their orientation and navigates through the map, the 3Dsounds (e.g., the sound of a waterfall) change their auditory positionrelative to the user. As the user moves through the map, sounds (e.g.,footstep sounds or spearcons) may be played based on the different dataassociated with the features present at the user's current position; 2)grid mode where a map is represented as a grid of tiles such that as auser moves into the tile, a spearcon and auditory icon may be playedrepresenting the changing features present in the tile; 3) tree modewhere the map is represented in parent child relationships using a setof menus that are read out in speech. Existing audio maps are typicallylimited in the modes through which a user may navigate the map. Forinstance, many times, a user is locked into a first-person mode.Additionally, auditory elements that may provide environmental cues andadded realism are often lacking from such maps.

Furthermore, existing maps may not provide users with options to selecttheir preferred mode of usage. For example, the existing maps may usetheir own text to speech engines (TTS), rather than allowing users touse their own screen readers.

SUMMARY OF THE INVENTION

A need exists for inexpensive digital nonvisual mapping. A need alsoexists for a digital nonvisual mapping solution that can representcomplex spatial information. A need exists for digital auditory mapsthat provide users with flexibility in navigating the environment tosuit a user's preferences or goals. Additionally, a need exists fordigital auditory maps that may provide auditory elements that mayprovide cues to a user that may aid in nonvisual navigation in acorresponding environment. Furthermore, a need exists for digitalauditory maps that may allow a user to incorporate the user's own screenreader and provide text to speech capability. Furthermore, a need existsfor auditory maps that can be applied to various platforms or useapplications (e.g., accessible through a browser).

In this present disclosure, a method is provided for a digital auditorymapping interface. The method comprises: accepting data representativeof spatial measurements of features within a real-world or data-basedenvironment; and generating a map comprising one or more points,polygons, or lines, representative of one or more spatial features. Themap comprises a plurality of navigational modes comprising: a tree-basedmode comprising of a menu of features accessible in a hierarchicalmanner; grid mode comprising auditory feedback whenever a user enters anew grid tile, the speech feedback comprising at least the name of oneor more of the features navigated to by the user, along with a soundrepresenting one or more of the properties of the features under theuser; and a first person mode configured to enable navigation andorientation changes in a selected direction at a specified rate,comprising one or more auditory cues of the navigation and surroundingobjects. Using this system, a digital map representative of anenvironment may be provided with geometries representative of featureswithin the environment. Other representational modes can include aheatmap where the features are represented by auditory queues derivedfrom statistical properties. The Javascript function to accept theuser's input to select the navigation mode is called:handleUserInput(event) as disclosed in the code listing in Appendix 1.

The digital auditory map may allow a user to access the map via atree-based mode, grid mode, or first person mode. A user may be able toselect their own speech preferences.

Another aspect of the present disclosure provides a non-transitorycomputer readable medium comprising machine executable code that, uponexecution by one or more computer processors, implements any of theoperations above or elsewhere herein.

Additional aspects and advantages of the present disclosure will becomereadily apparent to those skilled in this art from the followingdetailed description, wherein only illustrative embodiments of thepresent disclosure are shown and described. As will be realized, thepresent disclosure is capable of other and different embodiments, andits several details are capable of modifications in various obviousrespects, all without departing from the disclosure. Accordingly, thedrawings and descriptions are to be regarded as illustrative in nature,and not as restrictive.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings (also “figure” and “FIG.” herein) of which:

FIG. 1 shows an example of a representation of a polygon-based digitalauditory map, in accordance with embodiments of the invention.

FIG. 2A and FIG. 2B show an example of a user navigating an environmentwith aid of a digital auditory map, in accordance with embodiments ofthe invention.

FIG. 3 shows examples of information that may be integrated or used toformulate a digital auditory map, in accordance with embodiments of theinvention.

FIG. 4 shows an example of a tree-based mode of a digital auditory map,in accordance with embodiments of the invention.

FIG. 5 shows an example of a grid-based mode of a digital auditory map,in accordance with embodiments of the invention.

FIG. 6 shows an example of a first-person-based mode of a digitalauditory map, in accordance with embodiments of the invention.

FIG. 7 shows a representation of varying zoom levels, in accordance withembodiments of the invention.

FIG. 8 shows examples of how polygonal representations on the digitalmap may have permeable or solid boundaries, in accordance withembodiments of the invention.

FIG. 9 shows a flow chart of an exemplary process for user to select anduse different auditory options, in accordance with embodiments of theinvention.

FIG. 10 shows an exemplary computer system, in accordance withembodiments of the invention.

DETAILED DESCRIPTION

While various embodiments of the invention have been shown and describedherein, it will be obvious to those skilled in the art that suchembodiments are provided by way of example only. Numerous variations,changes, and substitutions may occur to those skilled in the art withoutdeparting from the invention. It should be understood that variousalternatives to the embodiments of the invention described herein may beemployed.

The invention provides systems and methods for digital auditory mapping.Various aspects of the invention described herein may be applied to anyof the particular applications set forth below. The invention may beapplied as a part of a mapping system or site that disseminatesinformation about an environment. It shall be understood that differentaspects of the invention can be appreciated individually, collectivelyor in combination with each other.

The term “digital map” as utilized herein, generally refers to a dynamicrepresentation of items configured in spatial and topological relationswith each other, represented in a virtual sensory format.

The term “screen reader” as utilized herein, may refer to softwareapplications that translate textual and graphical information displayedon a screen and re-present it to the user using synthesized speech.Screen readers are a form of assistive technology (AT) potentiallyuseful to the blind, visually impaired, color blind, low vision,dyslexic, illiterate or learning disabled, often in combination withother ATs such as screen magnifiers and tools for manipulating fonttype, font size, contrast, and the like.

The term “Text to Speech (TTS)” as utilized herein, may refer tosynthesized speech either generated through a speech API, or through ascreen reader.

The term “geometry” as utilized herein, may refer to collections ofpoints and vectors, for specifying geometricsA such as points, polygons,and lines, or collections of points, polygons, and lines.

The term “feature” as utilized herein, may refer to an object, datapoint, or any element present in a spatial map that consists ofproperties and geometries. It may also refer to any combination of theabove.

The term “data-based” as utilized herein, may refer to an applicationthat represents a collection of features representing a real-worldenvironment, simulated environment, imaginary environments, or othersets of data that have one or more features.

The term “auditory icon” as utilized herein, may refer to a soundrecorded from, or simulating the real-world environment, and thatrepresents data through one or more auditory elements, such as pitch,volume, tambour, rhythm, or duration. For example, a footstep on awooden surface representing a medium sized person with a hard-soled shoewalking on a wood surface IN A BUILDING.

The term “earcon” as utilized herein, may refer to a sound, or groupingof sounds that provide a symbolic representation of data. For example,the ‘ding’ when one receives a text message.

The term “spearcon” as utilized herein, may refer to a short messageplayed through TTS.

The term “sonification” as utilized herein, may refer to using sound torepresent data. Sonification may use non-speech audio to conveyinformation or perceptualize data.

As utilized herein, terms “component,” “system,” “interface,” “unit” andthe like are intended to refer to a computer-related entity, hardware,software (e.g., in execution), and/or firmware. For example, a componentcan be a processor, a process running on a processor, an object, anexecutable, a section of a web page, a program, a storage device, and/ora computer. By way of illustration, an application running on a serverand the server can be a component. One or more components can residewithin a process, and a component can be localized on one computerand/or distributed between two or more computers.

Further, these components can execute from various computer readablemedia having various data structures stored thereon. The components cancommunicate via local and/or remote processes such as in accordance witha signal having one or more data packets (e.g., data from one componentinteracting with another component in a local system, distributedsystem, and/or across a network, e.g., the Internet, a local areanetwork, a wide area network, etc. with other systems via the signal).

Moreover, the word “exemplary” is used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “exemplary” is not necessarily to be construed as preferred oradvantageous over other aspects or designs. Rather, use of the wordexemplary is intended to present concepts in a concrete fashion. As usedin this application, the term “or” is intended to mean an inclusive “or”rather than an exclusive “or.” That is, unless specified otherwise, orclear from context, “X employs A or B” is intended to mean any of thenatural inclusive permutations. That is, if X employs A; X employs B; orX employs both A and B, then “X employs A or B” is satisfied under anyof the foregoing instances. In addition, the articles “a” and “an” asused in this application and the appended claims should generally beconstrued to mean “one or more” unless specified otherwise or clear fromcontext to be directed to a singular form.

Digital auditory maps are usually provided as a single platform withbuilt-in voicing features (e.g., self-voicing). This may require usersto practice in order to get familiar with the default voicing feature.The digital auditory map of the present disclosure may advantageouslyallow users to utilize their own preferred screen reader or their ownspeech engines. For example, the digital auditory map may be provided asa web browser-based application utilizing the web speech API for text tospeech, or the user's existing screen reader through ARIA live regions.

Various aspects of the present disclosure may be applied to any of theparticular applications set forth below or for any other types ofapplications or systems. Systems or methods of the present disclosuremay be employed in a standalone manner, or as part of a package.

FIG. 1 shows an example of a representation of a digital auditory map100, in accordance with embodiments of the invention. In someembodiments, the digital auditory map 100 may include a geometries-basedmap where physical objects in the real-world may be represented by 2Dpolygonal regions, points, and/or lines, within a map. The digitalauditory map may represent a real physical environment a user may wishto explore virtually. A user may be blind, visually impaired, colorblind, low vision, dyslexic, illiterate, learning disabled, using avoice assistant or other individuals who may wish to utilize a nonvisualmap.

The environment may include an indoor environment, outdoor environment,fantasy environment, or a combination of the above. Examples ofenvironments may include, but are not limited to, playgrounds, parks,indoor play structures, shopping malls, airports, stores,schools/campuses, sporting arenas, convention centers, performing artscenters, museums, zoos, aquariums, continents, cities, planets, literaryenvironments, or any other types of space or structures. The environmentmay comprise any type of area where taking a virtual tour or viewing avisual map ahead of time or onsite may be useful. This may include areasof any size (e.g., large regions, medium sized regions, or smallerregions). This may include areas of varying levels of traffic ordensity. The environment may include one or more physical objects orstructures (e.g., path, bridge, long ramp, roller slide, swing, etc.)and the like a user may walk over or interact with. The one or morephysical objects or structures may be formed from varying materials. Theone or more physical objects may optionally emit a sound when interactedwith by a user or other individuals within the environment. For example,an environment may include road segments/paths, land cover such asbodies of water (e.g., rivers, oceans, lakes, swimming pools, etc.),administrative bodies (e.g., boundaries of states, countries, cities,parks, etc.), area designations (e.g., rural/urban/suburban,desert/mountains/forest, etc.), buildings, facilities, and variousothers. For example, as illustrated in FIG. 1, the data-based map mayillustrate a playground map and the polygons may represent variousobjects or structures such as 1=Ava's Bridge, 2=Climbing giraffe,3=creek bridge, 4=KinderBells, 5=long ramp, 6=roller slide, 7=steppingsounds.

The data-based map may be generated based on map service data receivedfrom one or more sources (e.g., mapping applications, vendors) or mapservice. A map service may provide map information and other map-relateddata, such as two-dimensional map vector data, three-dimensional mapvector data (e.g., traversable map with three-dimensional features, suchas buildings), route and direction calculation (e.g., directions betweentwo points for a pedestrian), real-time navigation data (e.g.,tum-by-tum navigation data in two or three dimensions), location data,physical measurements (e.g., size of building or structures) and othergeographic data (e.g., wireless network coverage, floor map, weather,traffic information, or nearby points-of-interest). In some cases, themap service data may also include localized labels for one or moreobjects (e.g., object names, points of interest, material or propertyattributes, coordinates, elevation, etc.). The one or more sources mayinclude internal or external sources. For example, vector data may beobtained from external services, or internal systems, or storagedevices. Other examples may include, but are not limited to, GPSassistance servers, wireless network coverage databases, geojson files,Open Street Map, business or personal directories, public sector (e.g.,visitor center of a park, playground website, etc.), weather data,government information (e.g., construction updates or road namechanges), or traffic reports. As described, the sources may also includephysical measurements relating to the environment from an in-personvisit. The physical measurements may optionally be used in combinationwith data from other mapping or data sources, such as satellite data orvoting statistics. The one or more sources may include publicly orprivately accessible data that may be accessed by a mapping system toaid in generation of the map layers.

The polygon map may be obtained directly from the aforementionedsources. Alternatively or additionally, at least a portion of thepolygon map may be generated by the mapping system. For instance, a maprendering component may render geometries for the objects (i.e., sets ofvertices and boundaries for drawing the objects) based on object graphsusing various processes. In some cases, the map rendering component mayperform various operations to refine the geometries such as smoothingtransitions between objects segments, creating more realistic roundedcomers at intersections of objects, or removing overlap between separateobjects that do not intersect. In some cases, the map renderingcomponent may use various operations to resolve boundaries between thegeometries. For example, when combining data from different sources, thelocation data indicating object boundaries may not align perfectly andtherefore there may be either gaps between the object geometries oroverlap of the geometries. The map rendering component may use differentoperations for resolving boundaries between different geometries,depending on the types of objects.

In addition, in some cases the map rendering component assigns specificproperties to the feature. For example, a boundary of a feature'spolygon can be set as a solid barrier such that when a user crosses overthe boundary auditory feedback (e.g., an auditory icon) may be deliveredto the user indicating the user collides with a feature. In anotherexample, the boundary of a polygon can be set as having a permeableboundary such that a user can enter or exit the polygon, and auditoryfeedback may be provided to the user indicating the name of the feature(or features) that the user enters and exits. In another example, thefeature may be connected with a data only layer and assigned anattribute, or set of attributes, such as number of people withdisabilities and or number of eligible voters.

The one or more features may be representative of physical real-worldobjects within the environment. The shape of the feature's geometries(e.g., points, lines, polygons) may be representative of the shape ofthe real-world object (although the shape of the geometry need notaccurately match the shape of the real-world object). A user may alsostep on/over or interact with the real-world object represented by thefeature. A geometry may also be provided for any other type of object,such as boundaries, and so forth.

A user may interact with the digital auditory map by navigating withinthe vector-based or geometries-based map and receive auditory feedback.In some cases, one or more of the polygonal regions may be associatedwith auditory data related to, for example, the name of the objectcorresponding to the polygonal region, the coordinates and various otherattributes. For instance, a polygon representing an object in aplayground (e.g., swing, swimming pool, long ramp, etc.) may store dataabout the name of the object, and the coordinates of the object. Inanother example, a polygon representing a road segment may store datadefining the location and properties of roads. For instance, a roadsegment is assigned one or more names (e.g., “long ramp”, “LincolnBlvd.”, “CA-I”), location data that indicates the path of the roadsegment or coordinates, and attributes of the road (e.g., surface type,materials, speed limit, width, number of lanes, road type, etc.). Inother examples, a building polygon may store the location of buildingsas well as data about the buildings. For instance, the building data mayinclude ground elevation data and surface elevation from which buildingheight may be calculated.

The auditory data may be played indicating the coordinates, location,name of the object and/or the sounds associated with the object. In somecases, the auditory data may be related to one or moreproperties/attributes of the object (e.g., materials, environment aroundthe object, size of the object, population of the object, functionalityof the object, etc.). The sounds associated with the object may berealistic sounds of the object or sonification of the numeric values. Insome examples, the sounds associated with the object may include soundsthat would be emitted from the object when a user or other individualswould interact with the object. The realistic sounds associated with theobject may include real-world sounds that were recorded for the objector within a close proximity of the object. For example, if the object is‘bells’, the realistic sounds may include recorded sounds of the bellsringing. In some instances, the sounds may include synthetic orcomputer-generated sounds that may be representative of the object. Forexample, the sound may be played based on the location in the digitalmap indicating the ambient environment at the location and/or soundsassociated with the object (e.g., swimming pool, playground object,etc.). In another example, a feature may have a numeric property, suchas population, and when a user enters the polygon of the feature, anearcon (e.g., a single pitch) is played representing the numeric data.The sounds may be captured from the real-world and/or synthesized.

In some embodiments, the sounds may be representative of an action to betaken in relation to the object and/or properties of the object. Forexample, if the object is a road and the user would typically step onthe road, the sounds may include the sound of footsteps on the road.Depending on a property of the road, such as the materials, differenttypes of footstep sounds may be played. For example, the sound ofwalking on wood may make a different sound compared to walking oncement.

In some cases, auditory data may be stored as a property of the featureobjects. The auditory data may be played when a user encounters apolygon region. For example, when a user enters or exits a boundary of afeature's polygon, sounds and TTS may be played indicating the name orother properties of the feature that the user enters or exits. Any otherspearcon or recorded sounds can be played.

In some cases, the auditory data may be played with respect to differentnavigational modes (e.g., first person mode, tree mode, grid mode) ofthe digital auditory map. For example, in a first-person mode, if theuser navigates into a polygonal region corresponding to an object suchas a long ramp having a material attribute as “wood”, sound mimickingfootsteps on wood materials may be played when the user “walks” over theobject. In another example, in the grid mode, when a user steps on atile within a portion of a polygon, an TTS spearcon indicating the nameof the object and coordinates of the object may be played. Details aboutthe different navigational modes are described later herein. TheJavascript function that accepts the user's input to select thenavigation mode is handleUserInput(event) as disclosed in the codelisting in Appendix 1.

The digital auditory map can be navigated or virtually explored atadjustable scale or zoom level. Both the polygon shapes, sizes and theassociated auditory data may be dynamically presented/played atdifferent zoom levels/scales of the digital auditory map. For example,shape of the polygon may be based on the area of the polygon divided bythe perimeter of the polygon, multiplied by a scaled factor. In anotherexample, users may experience with higher level of granularity, i.e.,greater detail, in terms of the shape of the polygon region as well asthe auditory sounds of the environment and footsteps when zoom in.Features may be filtered based on the zoom level. Details about thedifferent zoom levels are described later herein.

FIG. 2 shows an example of a user 210 navigating an environment 200 withaid of a digital auditory map 250, in accordance with embodiments of theinvention. A user may access the digital auditory map through a webbrowser (e.g., business website) to learn about an environment.

The user may access the digital auditory map at any location remote fromthe environment to be explored. For example, a user may access aplayground website to practice on the digital auditory map ahead of timein order to get familiar with the environment before visiting or whilenavigating the physical space. The user may access any site relating toan environment to be explored through a browser or application. The usermay be able to virtually explore an environment before physicallyvisiting an environment, which may advantageously provide the user withsome familiarity ahead of time. Accessing a digital auditory map aheadof time may allow the user to understand the spatial layout andassociated sounds in the space before physically venturing into thelocation. The digital auditory map may or may not have a correspondingvisual or other sensory representation that may be displayed on a userdevice.

Alternatively or in addition to, a user may use the digital auditory mapfor real-time navigation. For example, a user 210 may navigate theenvironment 200 with aid of the digital auditory map 250 in real-time.The user may have a portable user device 220 which may provide access tothe digital auditory map 250. The user device may be any type of devicesuch as a computing device configured to perform one or more operationsconsistent with the disclosed embodiments. Examples of user devices mayinclude, but are not limited to, mobile devices, smartphones/cellphones,tablets, personal digital assistants (PDAs), laptop or notebookcomputers, desktop computers, media content players, television sets,video gaming station/system, virtual reality systems, augmented realitysystems, microphones, or any electronic device configured to enable theuser to access the digital auditory map. The user device may be ahandheld object. The user device may be portable. The user device may becarried by a human user. In some cases, the user device may be locatedremotely from a human user, and the user can control the user deviceusing wireless and/or wired communications. The user may use the userdevice to access the digital auditory map at any point, such as prior toentering the environment, at an entrance to the environment, or whilethe user is navigating within the environment.

In the illustrated example, a real-time location of the user 210 may betracked by a user device 220. The real-time location of the user may bemapped to the current location 260 on the digital auditory map 250, andauditory feedback associated with the features 270, 280 may be playedwhen the user is detected to be entering a polygonal regioncorresponding to the features 220, 230. For example, when a location ofthe user is detected to be at a geo-boundary of the polygonal regionrepresenting the feature 230, a short auditory message about the name ofthe feature may be played.

In some embodiments, a user may be able to virtually navigate theenvironment while physically at or within the environment. The real-timelocation of the user may optionally be automatically updated in relationto the digital auditory map. In some instances, the real-time locationof the user may only be updated upon command from the user. For example,the user may be at a particular location within the environment, and theuser may input a command that may allow the virtual representation ofthe user to ‘jump’ to the same corresponding location within the map.From that location, the user may choose to virtually navigate within theenvironment without requiring the user to physically move within thespace. The location of the virtual representation of the user within thedigital auditory map need not directly correlate to the physicallocation of the user within the environment. The user may select anoption to synchronize the location of the virtual representation of theuser within the map with the physical location of the user in theenvironment. The user may be permitted to switch between a real-timenavigation mode based on physical location of the user and a virtualnavigation mode based on user-inputted location.

The real-world location of the user can be tracked by using any suitabledevices or methods. For example, the real-world location may be trackedusing locating component of the user device such as a global positioningsystem (GPS), a camera utilizing computer vision, or beacons. In somecases, differential global positioning system (DGPS) sensor and/or IMUmay be used to assist the user in navigating its environment anddetermining the orientation/position of the user with respect to aglobal reference frame. Any description herein of a DGPS sensor mayapply to any type of GPS sensor. The DGPS sensor can communicate withone or more ground based reference station and/or GPS satellites toobtain one or more GPS data signals. Location detection may occur inreference to GPS coordinates. The DGPS system may preferably use anetwork of fixed, ground-based reference stations to broadcast thedifference between the positions indicated by the GPS satellite systemsand the known fixed positions. The stations may broadcast the differencebetween the measured satellite pseudoranges and actual (internallycomputed) pseudoranges, and receiver stations may correct theirpseudoranges by the same amount. The DGPS sensor can utilize anysuitable GPS technology, such as differential GPS (DGPS) or real timekinematic (RTK) GPS. The GPS sensor can be configured to determine theposition of the user to any suitable level of accuracy, such asmeter-level accuracy (e.g., within 10 m, 5 m, 2 m, or 1 m of accuracy)or centimeter-level accuracy (within 500 cm, 200 cm, 100 cm, 50 cm, 20cm, 10 cm, or 5 cm of accuracy). Other location techniques such ascomputer vision and other location sensory (e.g., accelerometer,gyroscope and magnetometer) can also be used to track the user'smovement and/or location.

In some cases, other real-world locating methods or systems such asBeacon devices for indoor/outdoor positioning may be utilized. In somecases, the environment may be facilitated with beacons forindoor/outdoor position tracking, such as populating the indoor/outdoorspace with Bluetooth Low Energy (BLE) beacons or alternatively UWBanchors that transmit a continuous stream of packets that are picked upby a BLE transceiver or an UWB transceiver on the user device. Forinstance, with BLE, a position of the user device (e.g., mobile device,wearable devices) can be identified based on the proximity technology.The proximity technology may include a plurality of beacons distributedabout a premise through which an individual is located or to navigate.The mobile device may be BLE compatible so as to determine anindividual's relative physical location to a beacon. Based on rangingdata or approximate distance between user's device to each beacon alongwith the unique beacon's properties, different level of positioningaccuracy can be achieved. For instance, the proximity technology maydetermine the location of a mobile device based on a proximity estimateof signal strength emitting from a beacon. In addition, it can beenhanced with a beacon triangulation method to determine the (x, y, z)local map coordinates of an individual's position referencing theproximity of three or more beacons. The receiver can estimate itsposition using the average of x, y, z, which is the localizedcoordinates of a floor map for e.g. (x1, y1, z1), (x2, y2, z2) and (x3,y3, z3). The real-time locating system may employ any suitable rangingand/or angulating methods which may include, for example, angle ofarrival, angle of departure, line-of-sight, time of arrival, timedifference of arrival, two-way ranging, symmetrical double sided two wayranging, near-field electromagnetic ranging or any combination of theabove. The real-world locating system may utilize any suitabletechnologies to provide real-time locating, including but not limitedto, ultra-wideband (UWB) technologies, ultrasound-based RTLStechnologies, GPS-enabled RTLS, Wireless local area network, Bluetooth,and various other technologies to provide location tracking or proximitymeasurement. The accuracy may range from, for example, 0.1 m to 10 m.

The virtual location of the user within the digital auditory map may beprovided with any degree of precision and/or accuracy. As describedabove, a user may input a command that may allow a virtualrepresentation of the user to ‘jump’ to the same corresponding locationwithin the map. For example, the command may indicate a locationrelative to an object such as ‘north side of swimming pool,’ coordinatesof a location or a region within the map such as ‘south region of thepark.’ The user could also jump to a position within a feature, such asthe center.

FIG. 3 shows an example of an auditory map generation system 310configured to generate a digital auditory map, in accordance withembodiments of the invention. In some cases, the auditory map generationsystem 310 may generate a digital auditory map 370 based on map servicedata such as satellite or map data 320, physical measurement 330, andauditory data such as real-world sounds 340, auditory icons/spearcons350, and speech data 360.

In some embodiments, the auditory map generation system 310 may includea map rendering component to generate a polygon map. The polygon map canbe the same as the geometries-based map as described elsewhere herein.The map rendering component can be the same as the map renderingcomponent as described in FIG. 1. For example, the polygon map may begenerated based on map service data received from one or more sources(e.g., mapping applications, vendors) or map service. A map service mayprovide map information and other map-related data, such astwo-dimensional map vector data 320 (e.g., aerial view of a parkutilizing satellite imagery converted to numeric vectors),three-dimensional mesh data (e.g., traversable map withthree-dimensional features, such as buildings), physical measurements330, route and direction calculation (e.g., directions between twopoints for a pedestrian), real-time navigation data (e.g., turn-by-turnvisual navigation data in two or three dimensions), location data, andother geographic data (e.g., wireless network coverage, floor map,weather, traffic information, or nearby points-of-interest). In somecases, the map service data may also include localized labels ormetadata for one or more objects (e.g., object names, points ofinterest, attributes, coordinates, elevation, etc.). The one or moresources may include internal or external sources. For example, vectordata may be obtained from external services, or internal systems, orstorage devices. Other examples may include, but are not limited to, GPSassistance servers, wireless network coverage databases, business orpersonal directories, public section (e.g., visitor center of a park,etc.), weather data, government information (e.g., construction updatesor road name changes), or traffic reports.

As described above, the polygon-based map and its properties may beobtained directly from the abovementioned sources or generated by themap rendering component. The map rendering component may rendergeometries for the objects (i.e., sets of vertices and boundaries fordrawing the objects) based on object graphs using various processes. Forinstance, the map rendering component may perform various operations torefine the geometries such as smoothing transitions between objectssegments, creating more realistic rounded corners at intersections ofobjects, or removing overlap between separate objects that do notintersect. In some cases, the map rendering component may use variousoperations to resolve boundaries between the geometries. For example,when combining data from different sources, the location data indicatingobject boundaries may not align perfectly and therefore there may beeither gaps between the object geometries or overlap of the geometries.The system uses different operations for resolving boundaries betweendifferent geometries, depending on the types of objects.

The map rendering component may assign specific characteristics to thepolygon vertices and/or edges. For example, a boundary of a polygon canbe set as solid barrier such that when a user crosses over the boundaryauditory feedback (e.g., an auditory icon) may be delivered to the userindicating the user collides with the edge a polygon. In anotherexample, the boundary of a polygon can be set as a permeable boundarysuch that a user can enter or exit the polygon, and auditory feedbackmay be provided to the user indicating the name of the object that theuser enters and exits. The auditory data may be generated usingreal-world captured sounds 340, auditory icons/spearcons 350 or speech360.

The real-world captured sounds 340 may include sounds that wererecorded/captured at the real environment (e.g., playground). This maybeneficially provide a realistic experience that may help a user getfamiliar with the real environment. This may capture sounds associatedwith an object. This may capture sounds emitted by the object (e.g.,sounds of bells if the object is the bells) and/or sounds at anenvironment proximate to the object (e.g., sounds of children's voicesas is typically present near the bells).

The auditory icons 350 may include sounds representative of actions(e.g., footsteps make different sounds when on different types ofsurfaces/materials, or generic sounds of a user bumping into an object),and the like. The auditory icons 350 can include real-world recordedsounds or synthesized sounds.

The speech 360 may include text describing the object (e.g., name of theobject), coordinates, warning messages, information about entering orleaving objects, and the like that can be translated into audio speechusing the user's own screen reader, or a provided TTS engine.

The auditory data may be related to the object such as the name of theobject, the coordinates or locations and/or one or more attributes ofthe object. The auditory data may be played indicating the coordinates,location, name of the object and/or the real-world sounds associatedwith the object. In some cases, the auditory data may be related to oneor more properties/attributes of the object (e.g., materials,environment around the object, functionality of the object, etc.). Forexample, the sound may be played based on the location in the digitalmap indicating the ambient environment at the location and/or soundsassociated with the object (e.g., swimming pool, playground object,etc.).

In some cases, auditory data may be stored with the polygon objects. Theauditory data may be played when a user virtually encounters a polygonregion within the digital map or based on a real-time location of theuser. For example, when a user enters or exits a boundary of a polygonregion, a spearcon may be played indicating the name of the region thatthe user enters or exits, or a recorded sound associated with the objectmay be played.

In some cases, the auditory data may be played with respect to differentnavigational modes (e.g., first person mode, tree mode, grid mode) ofthe digital auditory map. For example, in a first-person mode, if theuser navigates into a polygonal region corresponding to an object suchas a long ramp having a material attribute as “wood”, sound mimickingfootsteps on wood materials may be played when the user “walks” over theobject. In another example, under the grid mode, when a user steps on atile with a portion of a polygon, an auditory feedback indicating thename of the object and coordinates of the object may be played. Inanother example, under the tree mode, when a user moves over an objectin the list, a sound may play representing a property of the object,along with the name of the object in speech.

A digital auditory map may have one or more navigational modes. In someembodiments, two or more, three or more, four or more, or greaternumbers of navigational modes may be provided. In some embodiments, adigital auditory map may comprise a first-person mode, a grid mode, atree-based mode or any combination of the above. The differentnavigational modes may efficiently assist users in navigating thedigital auditory map with an improved user experience. In someinstances, different navigational modes may suit different purposes. Forexample, a user may use the tree-mode to move between objects, use thegrid mode to get information about the shape of the object or spatialinformation between objects, and use the first-person mode to walkroutes between objects. Allowing for multiple navigational modes mayadvantageously allow users with different preferences to explore anenvironment in a personalized way that suits how they understand spatialrelations.

FIG. 4 shows an example of a tree-based mode 400 of a digital auditorymap, in accordance with embodiments of the invention. The tree-basedmode may beneficially allow users to quickly jump to an object ofinterest. The tree-based mode may allow users to quickly move betweenobjects (e.g., playground structures/features), navigate betweendifferent polygonal regions, and access detailed information associatedwith a selected object.

As illustrated in the example, the tree-based mode of the digitalauditory map may include a hierarchical structure illustratinghierarchical parent-child relationships between various objects andsettings within an environment. In some cases, the tree-based model mayinclude a menu listing regions/objects and options users can select, andchild menus with further options.

The tree-based mode can include menus with any number of levels (e.g.,at least one, two, three, four, or more levels). A user may be permittedto select an object at any level and to explore details (e.g., location,shape, properties, function, etc.) about the object. In some cases, uponselection of an object (e.g., slide, Merry-go-round, Swings, Ramp,etc.), a menu including one or more function options such as ‘go’,‘listen’, ‘description’, and ‘directions’ may be provided. The variousfunction options may allow a user to listen to the sounds associatedwith the selected object, learn the detailed description about theobject or directions to the object (e.g., location of the object orregion).

In some cases, an auditory spearcon about the name attribute of theobject may be played when the user moves through the menu. The ‘go’function may take the user to the center of the object polygon and theuser may virtually explore the environment starting from the center ofthe object polygon. The ‘listen’ function may permit the user to hearthe sound associated with the object in isolation from other sounds. Forexample, if the user selects the ‘listen’ function, the user may hearsound about the object and/or the environment within proximity of theobject. The ‘description’ function may permit users to hear the textualdescription of the object. The ‘directions’ function may permit the userto listen to the location of the object relative to the user's currentposition and the nearest point. In some cases, these functions may beprovided in response to a user input such as keyboard input. Forexample, a user may press Enter on each object to bring up the submenuincluding the abovementioned functions. The user may press a keyboardshortcut such as “d” to quickly replay updated directions to theselected object relative to the user's current location.

The tree-based mode may also permit users to configure one or moresettings of the digital auditory map. For example, a user may selectdifferent settings for virtual navigation (e.g., user preferred mode forvirtual navigation, default zoom-level for grid mode, etc.) andreal-time navigation (e.g., user preferred mode for real-timenavigation), settings for audio (e.g., screen reader, user preferredvoicing assistant, reading speed, volume, etc.) and various others.

FIG. 5 shows an example of a grid mode 500 of a digital auditory map, inaccordance with embodiments of the invention. The grid mode maybeneficially allow users to obtain shape information about object andspatial relationships between objects by sensing the boundary of apolygon region and distance between two polygon regions.

In a grid mode, the digital auditory map may be divided into a grid. Aunit within a grid may be a tile or square. Varying zoom levels may beprovided for the grid mode, which may provide a varying size of polygonand varying user movement.

In some cases, the unit of movement in the grid mode may be a tile orsquare. For example, if the user is represented by a single point, aunit of movement may move the user from [1,1] to [1,2] in a cartesianplane. Alternatively or additionally, a user may set up the unit ofmovement in the grid mode such as half tile or two tiles. As illustratedin the example, a user may input a user command (e.g., a keyboardcommand) and each movement step may be at the speed of a tile in thegrid. A user may be able to dictate how quickly the user travels withinthe grid mode by moving from a tile to adjacent tile on command. Thismay beneficially allow a user to get an overview of an environmentquickly. For example, a user may quickly press corresponding keys tonavigate the tiles of the grid, or to do an overall scan of theenvironment.

In some cases, the grid mode may provide detailed auditory feedback andspeech compared to the tree-based mode. For example, when a user movesto a new tile in the grid, a spearcon (e.g., short speech message) aboutthe name attribute of the polygon followed by the coordinates may beplayed. The coordinates may be, for example, the index of a tile (x, y)in the grid. The unit of the coordinates may be, for example, index of atile, meter, latitude, or longitude or others. In another example, whena user enters a tile with at least a portion of the polygon, an auditoryicon may be played. The auditory icon may be generated using real-worldrecordings of the object or synthesized sound. In some cases, thespearcon and the auditory icon may be played together.

In some cases, the orientation of the grid mode may be locked. When auser switch from the grid mode to the first-person mode, the orientationmay not change thereby preventing disorientation. In other cases,switching between first-person and grid mode may preserve the lastorientation the user had when they were in a mode.

FIG. 6 shows an example of a first-person mode 600 of a digital auditorymap, in accordance with embodiments of the invention. The first-personmode may beneficially provide detailed experience at a current locationof the user. For example, a user may be provided with positions ofobjects around through auditory icons, and/or footsteps that indicatethe type of terrain and walking speed thereby providing a realisticconnection to the real world.

In some cases, the first-person mode may have a user-selectedorientation. For example, a user may set up the top of a playground mapfacing the user as north. A user may change the orientation of thefirst-person mode at any time. In some cases, once the orientation isset, the orientation may be locked while the user is navigating thedigital map to prevent disorientation.

In some cases, a user may set up a moving speed navigating the digitalmap by changing the speed or size of the movement (e.g., footsteps). Forexample, a user may press and hold an arrow key to use the footsteps towalk a specified distance every 0.3 seconds. A default moving speed maybe provided. The moving speed may be increased or decreased by userpreference. The distance moved every step may be changed based on thezoom level.

As illustrated in the example, when the user enters a polygon, arecorded label may be played saying the name of the object. In somecases, if the terrain/material attribute of the object (e.g., wood) isavailable, the footsteps of the materials may be played when the userwalks over the object. In some cases, multiple objects may overlap suchthat at least one object may be on top of another. For example, objectscan be placed on, at, or within an object having a type attribute of“room”. In such cases, a phrase may be created, organizing the objectsat the user's location, based on the object's attributes. For example, aphrase may be “Entering Roller Slide at the Slide Mound” where theRoller Slide has a type attribute of “room” and Roller Slide has a typeattribute of “object”. The phrase may be created by a phrase creationalgorithm of the provided system.

The digital auditory map may permit users to seamlessly switch among themultiple navigational modes as described above. For example, a user mayswitch from a tree-based mode to the grid mode or first-person mode toexplore the shape and spatial information about an object selected inthe tree-based mode. The tree-based mode may allow a user to quickly‘jump’ to an object, while the grid mode or first-person mode may allowa user to explore the environment around the object. In another example,a user may toggle between a grid mode and a first-person mode to explorean environment at different granularity levels. The user may be able toscan through an environment quickly in grid mode and explore theenvironment in a more realistic setting by switching to first personmode, and vice versa. Providing multiple modes as provided may increasethe functionality of the digital auditory map and allow a user tovirtually tour the environment in a manner that is suited to the user'spreferences.

In some cases, when switching between a grid mode and a first-personmode, at least the footsteps/zoom level and/or orientation may bepreserved such that a user may not be disorientated. For example, when auser switches from a grid mode to a first-person mode, the speed andzoom level in the grid mode may be maintained and the user mayseamlessly continue navigating in the first-person mode at the samespeed, zoom level, and heading towards the same direction.

In some instances, when a user switches from the grid mode to thefirst-person mode, the user may remain oriented in the primary direction(same to the grid mode). In some cases, when the user switches from thefirst-person mode to the grid mode, the map may be automaticallyoriented to align to the primary direction of the grid mode if thefirst-person orientation was different.

As described above, the digital auditory map can be navigated at varyingzoom level/scales. FIG. 7 shows a representation of varying zoom levels,in accordance with embodiments of the invention. In some cases, at leastthe first-person mode and grid mode may have variable zoom levels. Auser may adjust the zoom level during navigation. In some cases,different zoom levels may include the vector-based map with graphspresented with corresponding scales and auditory data with differentdetails. The zoom factor can be in any range such as from 0.1× to 50×.

In both the illustrated grid mode and first-person mode, when a userzooms in (e.g., 2×), the step size may become smaller so the user canexperience with higher level of granularity the shape of the object, thegeographic locations and the corresponding auditory feedback. Zooming inmay have a similar effect as ‘shrinking’ a user within the environmentto allow for finer, more detailed, exploration. The sounds of thefootsteps (e.g., speed or frequency) relative to the ambient/backgroundenvironment may also change according to the zoom level.

In some embodiments, the digital auditory map may be designed withbarriers of objects to allow users to sense the shape of the geometryregion with improved efficiency. FIG. 8 shows examples of how polygonalrepresentations on the digital map may have permeable or solidboundaries 510, in accordance with embodiments of the invention. In someembodiments, the boundary of the polygons may act as a barrier such thatwhen a user hits/collides with the barrier, an auditory feedback may beplayed such that a user may rapidly sense the shape of the polygonalregion without entering or exiting the region.

As illustrated in the example 500 where the polygon-based map does nothave barriers, a user may not hear auditory feedback until the usercrosses over a boundary i.e., entering/exiting a polygon. As shown inthe example, this may take more steps and/or more time for a user tosense the shape of the polygon. In the polygon-based map with solidbarriers such as shown in the example 510, when a user hits a barrier insolid mode, there may be auditory cues (e.g., an auditory icon)indicating the user collides with a boarder. In such scenarios, the usermay sense the shape by repeatedly moving in the direction they are ableto go and colliding with the boarders of the geometry. Having theboarder reduces the number of key presses needed to explore a shape inhalf, because the user does not need to retrace their movement if theyexit the geometry. In some instances, auditory feedback may be providedwhen a user collides with a barrier in solid mode. In some cases, thesolid boundary can be detected by a distancing sensor that may alsoindicate the location of the user relative to the solid boundary in bothdirection and distancing. For example, a radar may be used that plays arepeated sound every specified number of degrees spanning an areasurrounding the user. If the radar does not sense a barrier, a gentlesound may be played indicating an empty space. When the radar hits abarrier, the sound may turn into a substantive beep in the direction ofthe barrier. In another example, different pitches may be played fordifferent directions where there is a barrier. For example, a barrier tothe left of the user may be specified by a solid tone of 261.6 HZ and abarrier in front of the player may be specified as 329.63 HZ. In somecases, multiple pitches may be played together indicating the user maybe surrounded by barriers in multiple directions (e.g., user is at acorner). The sound of the barrier tones may get louder as the user movescloser to the barrier.

In some instances, all of the geometries (e.g., polygons, points, lines,etc.) within a digital auditory map may be in permeable mode. In someinstances, all of the geometries within a digital auditory map may be ina solid mode. A user may optionally be presented with options to switchbetween permeable and solid modes based on user preferences. In someinstances, optionally, one or more geometries may be presented inpermeable mode while one or more other geometries may be presented in asolid mode. In one example, a polygon representative of an object thatmay be stepped on or over (e.g., path, ramp, etc.) may be presented in apermeable mode while an object that may extend vertically that a userwould not be able to step over (e.g., slide, swings, etc.) may bepresented in a solid mode. Input about object types may be provided wheninitially generating the digital auditory map, and the geometries mayhave different borders depending on object type.

The present digital auditory map system may advantageously permit usersto use their own screen reader to deliver the auditory feedback. In someembodiments, the digital auditory map system may be implemented as aweb-based application utilizing the Web Speech API to provide TTS, oruse an ARIA live region to conveniently interface with the user'sexisting screen reader. An ARIA live region is a simple mechanism fornotifying screen readers when content is updated on the page. Forexample, a screen reader can only focus on one part of a page at a time.If something changes elsewhere on the page, the screen reader user maybe oblivious to it. When the update takes place within an ARIA liveregion, a screen reader is automatically notified (wherever its focus isat the time), and it conveys the updated content to the user. Thedigital auditory map may be available for all major desktop browserslike Chrome, Firefox, Safari, Microsoft Edge and Opera, etc.

In some cases, the digital auditory map may be offered as a componentthat can be embedded into a third-party webpage. A user may be providedwith a uniform resource locator (URL) that directs to a version of themap of the present disclosure. The user can insert the URL into adesired webpage through an iframe.

In some cases, the digital auditory map may be installed as a packagethat can be accessed programmatically by other applications. Forexample, users may install the package from the Node Package Manager(NPM) and import the package into a desired webpage application. Theuser may then use functions, classes, and other features of the packagein the desired applications. In some cases, the package may requireauthentication and functionality through an external server.

In some cases, the digital auditory map may be a software applicationimplemented in a combination of one or more programming languages andmarkup languages for execution by various web browsers. For example, theclient software can be executed in web browsers that support JavaScriptand HTML rendering, such as Chrome, Mozilla Firefox, Safari, and anyother compatible web browsers. The various embodiments of softwareapplications may be compiled for various devices, across multipleplatforms, and may be optimized for their respective native platforms.

In some cases, the digital auditory map system may include a userinterface (UI) module. The user interface (UI) module may provide agraphical user interface (GUI), along with an auditory user interface(AUI) that can be integrated into other applications. A user may provideuser input (e.g., navigation in the map, interaction with a graphicalelement on the map, etc.) via the GUI, programmatically, through awebhook, or through an input device such as a keyboard. The userinterface may utilize one or more user input devices (e.g., mouse,joystick, keyboard, trackball, touchpad, button, verbal commands,gesture-recognition, attitude sensor, thermal sensor, touch-capacitivesensors, accelerometers, gyroscopes, speech recognition, or any XR(AR/VR/MR) devices).

FIG. 9 shows a flow chart of an exemplary process for user to select anduse different auditory options, in accordance with embodiments of theinvention. A user may access the digital auditory map 910 and may beprovided with options for the user to select a preferred auditoryinterface 920 for playing the audio. For example, a user may select asynthesizer from the Web Speech API 930, their screen reader 940, or anyother existing software synthesizers 950.

The Web Speech API is agnostic of the underlying speech recognition andsynthesis implementation and can support both server-based andclient-based/embedded recognition and synthesis. A Web Speech APItypically has two functions, speech synthesis, otherwise known as textto speech, and speech recognition. With the SpeechSynthesis API, abrowser is able to read out any text in a number of different voices. Auser may be permitted to select a user preferred voice within thebrowser enabled by the Web Speech API. As described above, screenreaders are software programs (e.g., Job Access With Speech, NonvisualDesktop Access, Voice Over, Talkback, ChromeVox, Speakup, Orea, etc.)that allow blind or visually impaired users to read the text that isdisplayed on the computer screen with a speech synthesizer or brailledisplay. Each screen reader may incorporate a different commandstructure, and most support a variety of speech synthesizers. A user maychoose to use screen reader or the Web Speech API based on theavailability of the software programs, preferred input methods (e.g.,keystrokes, voice commands, or gestures) or user preferred settings(e.g., voice).

A user may be permitted to choose their preferred voicing, language oraccent. For example, a user may choose an existing voice, gender,language, and accent from the options provided by the Web Speech API.

In some cases, two or more of the Web Speech API, Aria Live Regions,internally packaged synthesizer, recordings, or externally generatedspeech can be used simultaneously. For example, in the first-personmode, entering and exiting labels may be presented using the Web SpeechAPI, and all other speech may be presented using Aria live regions(through the user's own screen reader).

In some cases, the digital auditory map system may be implemented in adistributed computing environment, such as over the cloud. The digitalauditory map system may include services or applications that run in thecloud or an on-premises environment to perform one or more methodsconsisted with those described herein. This digital auditory map systemmay run in one or more public clouds (e.g., Amazon Web Services (AWS),Azure, etc.), and/or in hybrid cloud configurations where one or moreparts of the system run in a private cloud and other parts in one ormore public clouds.

The present disclosure provides computer systems that are programmed toimplement methods of the disclosure. FIG. 10 shows a computer system1001 that is programmed or otherwise configured to implement theweb-based digital auditory map. The computer system can be an electronicdevice of a user or a computer system that is remotely located withrespect to the electronic device. The electronic device can be a mobileelectronic device.

The computer system 1001 may include a central processing unit (CPU,also “processor” and “computer processor” herein) 1005, which can be asingle core or multi core processor, or a plurality of processors forparallel processing. The computer system also includes memory or memorylocation 1010 (e.g., random-access memory, read-only memory, flashmemory), electronic storage unit 1015 (e.g., hard disk), communicationinterface 1020 (e.g., network adapter) for communicating with one ormore other systems, and peripheral devices 1025, such as cache, othermemory, data storage and/or electronic display adapters. The memory1010, storage unit 1015, interface 1020 and peripheral devices 1025 arein communication with the CPU 1005 through a communication bus (solidlines), such as a motherboard. The storage unit 1015 can be a datastorage unit (or data repository) for storing data. The computer system1001 can be operatively coupled to a computer network (“network”) 1030with the aid of the communication interface 1020. The network 1030 canbe the Internet, an internet and/or extranet, or an intranet and/orextranet that is in communication with the Internet.

The network 1030 in some cases is a telecommunication and/or datanetwork. The network can include one or more computer servers, which canenable distributed computing, such as cloud computing. Such cloudcomputing may be provided by cloud computing platforms such as, forexample, Amazon Web Services (AWS), Microsoft Azure, Google CloudPlatform, and IBM cloud. The network, in some cases with the aid of thecomputer system 1001, can implement a peer-to-peer network, which mayenable devices coupled to the computer system to behave as a client or aserver.

The CPU 1005 can execute a sequence of machine-readable instructions,which can be embodied in a program or software. The instructions may bestored in a memory location, such as the memory 1010. The instructionscan be directed to the CPU, which can subsequently program or otherwiseconfigure the CPU to implement methods of the present disclosure.Examples of operations performed by the CPU can include fetch, decode,execute, and writeback.

The CPU 1005 can be part of a circuit, such as an integrated circuit.One or more other components of the system can be included in thecircuit. In some cases, the circuit is an application specificintegrated circuit (ASIC).

The storage unit 1015 can store files, such as drivers, libraries andsaved programs. The storage unit can store user data, e.g., userpreferences and user programs. The computer system 1001 in some casescan include one or more additional data storage units that are externalto the computer system, such as located on a remote server that is incommunication with the computer system through an intranet or theInternet.

The computer system 1001 can communicate with one or more remotecomputer systems through the network 1030. For instance, the computersystem can communicate with a remote computer system of a user (e.g., auser of an experimental environment). Examples of remote computersystems include personal computers (e.g., portable PC), slate or tabletPC's (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones(e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personaldigital assistants. The user can access the computer system via thenetwork.

Methods as described herein can be implemented by way of machine (e.g.,computer processor) executable code stored on an electronic storagelocation of the computer system 1001, such as, for example, on thememory 1010 or electronic storage unit 1015. The machine executable ormachine readable code can be provided in the form of software. Duringuse, the code can be executed by the processor 1005. In some cases, thecode can be retrieved from the storage unit and stored on the memory forready access by the processor. In some situations, the electronicstorage unit can be precluded, and machine-executable instructions arestored on memory.

The code can be configured to receive data representative of spatialmeasurements of features within a real-world or data-based environmentfrom storage 1015 or from external data sources obtained through thenetwork 1030 via the communication interface 1020. The code executed bythe processor 1005 can comprise a digital audio map component whichgenerates a digital audio map through a digital audio map generatorcomponent. The digital audio map component can present a digital audiomap to a user through a digital audio map presentation component. Thedigital map component may be written in a high level computer languagesuch as C/C++, C#, Visual Basic, Java, Ruby, Python, Go, Rust, andJavascript, including Javascript frameworks such as React and VueJS.

The code can be pre-compiled and configured for use with a machinehaving a processer adapted to execute the code, or can be compiledduring runtime. The code can be supplied in a programming language thatcan be selected to enable the code to execute in a pre-compiled oras-compiled fashion.

Aspects of the systems and methods provided herein, such as the computersystem 1001, can be embodied in programming. Various aspects of thetechnology may be thought of as “products” or “articles of manufacture”typically in the form of machine (or processor) executable code and/orassociated data that is carried on or embodied in a type of machinereadable medium. Machine-executable code can be stored on an electronicstorage unit, such as memory (e.g., read-only memory, random-accessmemory, flash memory) or a hard disk. “Storage” type media can includeany or all of the tangible memory of the computers, processors or thelike, or associated modules thereof, such as various semiconductormemories, tape drives, disk drives and the like, which may providenon-transitory storage at any time for the software programming. All orportions of the software may at times be communicated through theInternet or various other telecommunication networks. Suchcommunications, for example, may enable loading of the software from onecomputer or processor into another, for example, from a managementserver or host computer into the computer platform of an applicationserver. Thus, another type of media that may bear the software elementsincludes optical, electrical and electromagnetic waves, such as usedacross physical interfaces between local devices, through wired andoptical landline networks and over various air-links. The physicalelements that carry such waves, such as wired or wireless links, opticallinks or the like, also may be considered as media bearing the software.As used herein, unless restricted to non-transitory, tangible “storage”media, terms such as computer or machine “readable medium” refer to anymedium that participates in providing instructions to a processor forexecution.

Hence, a machine readable medium, such as computer-executable code, maytake many forms, including but not limited to, a tangible storagemedium, a carrier wave medium or physical transmission medium.Non-volatile storage media include, for example, optical or magneticdisks, such as any of the storage devices in any computer(s) or thelike, such as may be used to implement the databases, etc. shown in thedrawings. Volatile storage media include dynamic memory, such as mainmemory of such a computer platform. Tangible transmission media includecoaxial cables; copper wire and fiber optics, including the wires thatcomprise a bus within a computer system. Carrier-wave transmission mediamay take the form of electric or electromagnetic signals, or acoustic orlight waves such as those generated during radio frequency (RF) andinfrared (IR) data communications. Common forms of computer-readablemedia therefore include for example: a floppy disk, a flexible disk,hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD orDVD-ROM, any other optical medium, punch cards paper tape, any otherphysical storage medium with patterns of holes, a RAM, a ROM, a PROM andEPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wavetransporting data or instructions, cables or links transporting such acarrier wave, or any other medium from which a computer may readprogramming code and/or data. Many of these forms of computer readablemedia may be involved in carrying one or more sequences of one or moreinstructions to a processor for execution.

The computer system 1001 can include or be in communication with anelectronic display 1035 that comprises a user interface (UI) 1040 forproviding, for example, selection of an environment, a component of anenvironment, or a time point of an environment. Examples of UI'sinclude, without limitation, a graphical user interface (GUI), AuditoryUser Interface (AUI), and web-based user interface.

Methods and systems of the present disclosure can be implemented by wayof one or more algorithms. An algorithm can be implemented by way ofsoftware upon execution by the central processing unit 1005. Thealgorithm can, for example, capture a configuration of one or moreexperimental environments; store in a registry the experimentalenvironments at each of one or more time points; perform one or moreexperimental executions which leverage experimental environments;provide outputs of experimental executions which leverage theenvironments; generate a plurality of linkages between the experimentalenvironments and the experimental executions; and generate one or moreexecution states corresponding to the experimental environments at oneor more time points.

The digital auditory map is a software application running on acomputer. It could be distributed over a network and downloaded on theuser's machine, downloaded as a package to be programmaticallyinterfaced by other applications, accessed through a URL as either anAPI or embedded application, compiled into other applications,downloaded as a stand-alone application, or executed by a computingdevice that can run the code.

The computer system may be in communication with one or more databases.The one or more databases may utilize any suitable database techniques.For instance, structured query language (SQL) or “NoSQL” database may beutilized for storing the map data, auditory data, user information andthe like. Some of the databases may be implemented using variousstandard data-structures, such as an array, hash, (linked) list, struct,structured text file (e.g., XML), table, JSON, NOSQL and/or the like.Such data-structures may be stored in memory and/or in (structured)files. In another alternative, an object-oriented database may be used.Object databases can include a number of object collections that aregrouped and/or linked together by common attributes; they may be relatedto other object collections by some common attributes. Object-orienteddatabases perform similarly to relational databases with the exceptionthat objects are not just pieces of data but may have other types offunctionality encapsulated within a given object. If the database of thepresent invention is implemented as a data-structure, the use of thedatabase of the present invention may be integrated into anothercomponent such as the component of the present invention. Also, thedatabase may be implemented as a mix of data structures, objects, andrelational structures.

Databases may be consolidated and/or distributed in variations throughstandard data processing techniques. Portions of databases, e.g.,tables, may be exported and/or imported and thus decentralized and/orintegrated.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. It is not intendedthat the invention be limited by the specific examples provided withinthe specification. While the invention has been described with referenceto the aforementioned specification, the descriptions and illustrationsof the embodiments herein are not meant to be construed in a limitingsense. Numerous variations, changes, and substitutions will now occur tothose skilled in the art without departing from the invention.Furthermore, it shall be understood that all aspects of the inventionare not limited to the specific depictions, configurations or relativeproportions set forth herein which depend upon a variety of conditionsand variables. It should be understood that various alternatives to theembodiments of the invention described herein may be employed inpracticing the invention. It is therefore contemplated that theinvention shall also cover any such alternatives, modifications,variations or equivalents. It is intended that the following claimsdefine the scope of the invention and that methods and structures withinthe scope of these claims and their equivalents be covered thereby.

What is claimed is:
 1. A method for providing digital auditory mapping,said method comprising: accepting data representative of spatialmeasurements of features within a real-world or data-based environment;and generating a digital audio map comprising objects from the datarepresentative of one or more features, wherein the digital audio mapcomprises a plurality of navigational modes comprising one or more of: atree-based mode comprising an object menu accessible in a hierarchicalmanner, the object menu comprising objects within the environment; agrid-based mode comprising auditory feedback whenever a user enters anew grid tile containing one of the objects; and a first-person-basedmode configured to enable navigation at a selected direction at aspecified rate, comprising one or more auditory cues of the navigationand surrounding objects.
 2. The method of claim 1, further comprisingaccepting input from the user to switch between navigational modes;wherein when a user switches between grid-based mode andfirst-person-based mode, the speed, zoom level and/or orientation arepreserved.
 3. The method of claim 1, wherein the data representative ofspatial measurements of features within a real-world is accepted from anexternal data source.
 4. The method of claim 1, wherein the datarepresentative of spatial measurements of features within a real-worldis accepted from a plurality of data sources, which may be internal orexternal, and generating the digital audio map further comprisesresolving boundaries between any different geometries or relationshipsprovided by the plurality of data sources, and further comprisingdynamically adjusting the scale of the digital audio map whereby theobjects may be filtered based on the level of granularity, and theauditory data may be proportionately adjusted by conversion to adifferent coordinate system.
 5. The method of claim 1, furthercomprising accepting an input from the user to move to a specificlocation in the digital audio map.
 6. The method of claim 1, whereinspecific vertices and/or edges of specific features can be set to solidor permeable.
 7. The method of claim 1, further comprising accepting areal-time location of the user from a portable user device and mappingthe user to a current location in the digital audio map, wherein theuser's location can be updated automatically or manually by acceptinginput from the user.
 8. The method of claim 1, wherein generating thedigital audio map further comprises assigning specific properties to thefeatures, wherein the specific properties assigned to the featuresinclude auditory data representing contexts such as coordinates,location, name of the feature, and/or sounds associated with thefeature, which may be accessed by a user navigating to the feature. 9.The method of claim 8, wherein each of the contexts of the auditory dataare associated with one or more of the plurality of navigational modeswhereby the auditory data accessed by the user navigating to the featureis the context associated with the navigation mode being used by theuser.
 10. The method of claim 1, further comprising a listen functionproviding the auditory data associated with a selected object inisolation from the auditory data associated with the unselected objects.11. A computer system for providing digital auditory mapping, the systemcomprising: at least one processing unit; memory operably associatedwith the at least one processing unit; a digital audio map componentstorable in memory and executable by the at least one processing unit,the digital audio map program comprising: a digital audio map generatorcomponent configured to receive data representative of spatialmeasurements of features within a real-world or data-based environmentand generate a digital audio map comprising objects from the datarepresentative of one or more features; a digital audio map presentationcomponent configured to provide a plurality of navigational modescomprising one or more of: a tree-based mode comprising an object menuaccessible in a hierarchical manner, the object menu comprising objectswithin the environment; a grid-based mode comprising auditory feedbackwhenever a user enters a new grid tile containing one of the objects;and a first-person-based mode configured to enable navigation at aselected direction at a specified rate, comprising one or more auditorycues of the navigation and surrounding objects.
 12. The system of claim11, wherein the digital audio map component is further configured toaccept input from the user to switch between navigational modes; whereinwhen a user switches between grid-based mode and first-person-basedmode, the speed, zoom level and/or orientation are preserved.
 13. Thesystem of claim 11, wherein the digital audio map component is furtherconfigured to accept the data representative of spatial measurements offeatures from an external data source.
 14. The system of claim 11,wherein the digital audio map component is further configured to acceptthe data representative of spatial measurements of features from aplurality of data sources, which may be internal or external, andgeneration of the digital audio map further comprises resolvingboundaries between any different geometries or relationships provided bythe plurality of data sources, and further comprising dynamicallyadjusting the scale of the digital audio map whereby the objects mayfiltered based on the level of granularity, and the auditory data isproportionately adjusted by conversion to a different coordinate system.15. The system of claim 11, wherein the digital audio map component isfurther configured to accept an input from the user to move to aspecific location in the digital audio map.
 16. The system of claim 11,wherein the digital audio map component is further configured to accepta real-time location of the user from a portable user device and mappingthe user to a current location in the digital audio map, wherein theuser's location can be updated automatically or manually by acceptinginput from the user.
 17. The system of claim 11, wherein the digitalaudio map component is further configured to generate the digital audiomap further comprising assigning specific properties to the features,wherein the specific properties assigned to the features includeauditory data representing contexts such as coordinates, location, nameof the feature, and/or sounds associated with the feature, which may beaccessed by a user navigating to the feature.
 18. The system of claim11, wherein the digital audio map component is further configured suchthat each of the contexts of the auditory data are associated with oneor more of the plurality of navigational modes whereby the auditory dataaccessed by the user navigating to the feature is the context associatedwith the navigation mode being used by the user.
 19. The system of claim11, wherein the digital audio map component is further configured toprovide a listen function providing the auditory data associated with aselected object in isolation from the auditory data associated with theunselected objects.
 20. A computer-readable medium storing computerinstructions, which when executed, enables a computer system to providedigital audio map, the computer instructions comprising: accepting datarepresentative of spatial measurements of features within a real-worldor data-based environment; and generating a digital audio map comprisingobjects from the data representative of one or more features, whereinthe digital audio map comprises a plurality of navigational modescomprising one or more of: a tree-based mode comprising an object menuaccessible in a hierarchical manner, the object menu comprising objectswithin the environment; a grid-based mode comprising auditory feedbackwhenever a user enters a new grid tile containing one of the objects;and a first-person-based mode configured to enable navigation at aselected direction at a specified rate, comprising one or more auditorycues of the navigation and surrounding objects.